Search for: All records

Abstract In the realm of robotics and automation, robot teleoperation, which facilitates human–machine interaction in distant or hazardous settings, has surged in significance. A persistent issue in this domain is the delays between command issuance and action execution, causing negative repercussions on operator situational awareness, performance, and cognitive load. These delays, particularly in long-distance operations, are difficult to mitigate even with the most advanced computing advancements. Current solutions mainly revolve around machine-based adjustments to combat these delays. However, a notable lacuna remains in harnessing human perceptions for an enhanced subjective teleoperation experience. This paper introduces a novel approach of sensory manipulation for induced human adaptation in delayed teleoperation. Drawing from motor learning and rehabilitation principles, it is posited that strategic sensory manipulation, via altered sensory stimuli, can mitigate the subjective feeling of these delays. The focus is not on introducing new skills or adapting to novel conditions; rather, it leverages prior motor coordination experience in the context of delays. The objective is to reduce the need for extensive training or sophisticated automation designs. A human-centered experiment involving 41 participants was conducted to examine the effects of modified haptic cues in teleoperations with delays. These cues were generated from high-fidelity physics engines using parameters from robot-end sensors or physics engine simulations. The results underscored several benefits, notably the considerable reduction in task time and enhanced user perceptions about visual delays. Real-time haptic feedback, or the anchoring method, emerged as a significant contributor to these benefits, showcasing reduced cognitive load, bolstered self-confidence, and minimized frustration. Beyond the prevalent methods of automation design and training, this research underscores induced human adaptation as a pivotal avenue in robot teleoperation. It seeks to enhance teleoperation efficacy through rapid human adaptation, offering insights beyond just optimizing robotic systems for delay compensations. 

IntroductionLong-distance robot teleoperation faces high latencies that pose cognitive challenges to human operators. Latency between command, execution, and feedback in teleoperation can impair performance and affect operators’ mental state. The neural underpinnings of these effects are not well understood. MethodsThis study aims to understand the cognitive impact of latency in teleoperation and the related mitigation methods, using functional Near-Infrared Spectroscopy (fNIRS) to analyze functional connectivity. A human subject experiment (n= 41) of a simulated remote robot manipulation task was performed. Three conditions were tested: no latency, with visual and haptic latency, with visual latency and no haptic latency. fNIRS and performance data were recorded and analyzed. ResultsThe presence of latency in teleoperation significantly increased functional connectivity within and between prefrontal and motor cortexes. Maintaining visual latency while providing real-time haptic feedback reduced the average functional connectivity in all cortical networks and showed a significantly different connectivity ratio within prefrontal and motor cortical networks. The performance results showed the worst performance in the all-delayed condition and best performance in no latency condition, which echoes the neural activity patterns. ConclusionThe study provides neurological evidence that latency in teleoperation increases cognitive load, anxiety, and challenges in motion planning and control. Real-time haptic feedback, however, positively influences neural pathways related to cognition, decision-making, and sensorimotor processes. This research can inform the design of ergonomic teleoperation systems that mitigate the effects of latency.